aws_sign_string

The aws_sign_String statement is used to generate an AWS4-HMAC-SHA256 signature, used as the signature component of the Authorization HTTP header when calling the AWS API.

Syntax

aws_sign_stringvarNameusingsecret_key date region service

Details

The authentication method used by AWS requires the generation of an authorization signature which is derived from a secret key known to the client along with specific elements of the query being made to the API.

This is a fairly involved process and a full step-by-step walkthrough is provided by Amazon on the following pages (these should be read in the order listed below):

• https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
• https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html
• https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
• https://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html

The aws_sign_string statement is used to generate the final signature as detailed on the calculate signature page listed above.

Note that in order to use this statement it is necessary to have the following strings available:

1. A string to sign, obtained by following the process of creating a string to sign, containing meta-data about the request being made
2. A secret_key, obtained from Amazon which is used by any client application authorizing against their API
3. The date associated with the API request, in YYYYMMDD format
4. The AWS region associated with the API request (for example eu-central-1)
5. The AWS service being accessed (for example s3)

The aws_sign_string statement will use these inputs to generate the HMAC-SHA256 signature which is a component of the Authorization header when connecting to the API itself.

The varName parameter is the name of a variable containing the string to sign. After executing aws_sign_string the contents of this same variable will have been updated to the base-16 encoded signature value.

caution

If there are any errors in the string to sign, _date, AWS region or AWS service strings used as input to aws_sign_string then a signature will still be generated, but the AWS API will reject the request. In this case it is necessary to review the process by which these strings were created as per the AWS guide provided above.

Example

The following is an example USE script that implements everything described above.

#################################################################
# This USE script will download a file from an S3 bucket        #
#                                                               #
# It takes three parameters:                                    #
# 1) The name of the bucket                                     #
# 2) The name of the object to download                         #
# 3) The name of the file to save the downloaded object as      #
#                                                               #
# Created: 13th Jan 2018                                        #
# Author: Eddy Deegan                                           #
# --------------------------------------------------------------#
# NOTES:                                                        #
# - This script hardcodes the Region as eu-central-1 but this   #
#   can easily be changed or made a parameter as required       #
#################################################################

if (${ARGC} != 3) {
    print This script requires the following parameters:
    print bucketName objectName saveFilename
    terminate
}

# Set this to 1 to enable a debug trace output when the script is run
var DEBUG = 0

# This is the text that appears to the left and right of debug headings 
var banner = ________

######################################################################
# Customer specific values here (these can be encrypted if required) #
#                                                                    #
var bucket = "${ARG_1}"
var s3_object = "${ARG_2}"
var AWS_Region = "eu-central-1"
var AWS_Service = "s3"
encrypt var access_key = <YOUR ACCESS KEY>
encrypt var secret_key = <YOUR SECRET KEY>
#                                                                    #
# End customer specific values                                       #
######################################################################

# This is the SHA256 hash of an empty string (required if making a request with no body)
var hashed_empty_string = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

#########################################################################################
# SETUP                                                                                 #
# Create a number of variables to represent the various components that the steps       #
# below are going to use in order to construct a correct AWS request                    #
#---------------------------------------------------------------------------------------#
# This is the request syntax for retrieving an object from a bucket:                    #
# GET /<ObjectName> HTTP/1.1                                                            #
# Host: <BucketName>.s3.amazonaws.com                                                   #
# Date: date                                                                            #
# Authorization: authorization string                                                   #
#########################################################################################

var HTTP_Method = GET
var URI = ${s3_object}
var query_params                    # Must have an empty variable for 'no query parameters'
var host = ${bucket}.s3-${AWS_Region}.amazonaws.com
var date = ${OSI_TIME_UTC}

# Initialise config variables specific to this script
var save_path = "system/extracted"
var save_file = ${ARG_3}

#########################################################################################
# STEP 1                                                                                #
# Create a canonical request as documented at                                           #
# at https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html  #
#########################################################################################

# 1a) Canonical Headers string
#     - This is part of the Canonical Request string which will be generated below.
#     - The Canonical Headers are a list of all HTTP headers (including values but
#       with the header names in lowercase) separated by newline characters and in
#       alphabetical order

var canonical_headers = "date:${date}${NEWLINE}host:${host}${NEWLINE}x-amz-content-sha256:${hashed_empty_string}${NEWLINE}"
if (${DEBUG} == 1) {
    print ${NEWLINE}${banner} Canonical Headers ${banner}${NEWLINE}${canonical_headers}
}

# 1b) Signed Headers string
#     - This is a list of the header names that were used to create the Canonical Headers,
#       separated by a semicolon
#     - This list MUST be in alphabetical order
#     - NOTE: There is no trailing newline on this variable (we need to use it both with and without
#             a newline later so we explicitly add a ${NEWLINE} when we need to)

var signed_headers = "date;host;x-amz-content-sha256"
if (${DEBUG} == 1) {
    print ${banner} Signed Headers ${banner}${NEWLINE}${signed_headers}${NEWLINE}
}

# 1c) Canonical Request
#     - The above are now combined to form a Canonical Request, which is created as follows:
#     - HTTPRequestMethod + '\n' + URI + '\n' + QueryString + '\n' + CanonicalHeaders + '\n' +
#       SignedHeaders + '\n' + Base16 encoded SHA256 Hash of any body content
##    - Note that the Canonical Headers are followed by an extra newline (they have one already)

vvar canonical_request = "${HTTP_Method}${NEWLINE}/${URI}${NEWLINE}${query_params}${NEWLINE}${canonical_headers}${NEWLINE}${signed_headers}${NEWLINE}${hashed_empty_string}"
iif (${DEBUG} == 1) {
    print ${banner} Canonical Request ${banner}${NEWLINE}${canonical_request}${NEWLINE}
}}

## 1d) Hash of the Canonical Request
##     - This is an SHA256 hash of the Canonical Request string

hhash sha256 canonical_request as hashed_canonical_request

######################################################################################
## STEP 2                                                                             #
## Create a 'string to sign' as documented at                                         #
## at https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html  #
##------------------------------------------------------------------------------------#
## In a nutshell this is the following components separated by newlines:              #
## 2a) Hash algorithm designation                                                     #
## 2b) UTC date in YYYYMMDD'T'HHMMSS'Z' format                                        #
## 2c) credential scope (date/region/service/"aws4_request")                          #
## 2d) base16-encoded hashed canonical request                                        #
######################################################################################

## Extract the yyyyMMdd from the UTC time
mmatch yyyyMMdd "(.{8})" ${date}
vvar yyyyMMdd = ${yyyyMMdd.RESULT}

vvar string_to_sign = AWS4-HMAC-SHA256${NEWLINE}${date}${NEWLINE}${yyyyMMdd}/${AWS_Region}/${AWS_Service}/aws4_request${NEWLINE}${hashed_canonical_request}
iif (${DEBUG} == 1) {
    print ${banner} String to sign ${banner}${NEWLINE}${string_to_sign}${NEWLINE}
}}

######################################################################################
## STEP 3                                                                             #
## Calculate the signature for AWS Signature Version 4 as documented at:              #
## at https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html    #
##                                                                                    #
######################################################################################

## 3a) Derive a signing key and apply it to the string to sign
##     Use the secret access key to create the following hash-based auth codes:
##     a) ksecret (our secret access key)
##     b) kDate = HMAC("AWS4" + kSecret, Date) NOTE: yyyyMMdd only
##     c) kRegion = HMAC(kDate, Region)
##     d) kService = HMAC(kRegion, Service)
##     e) kSigning = HMAC(kService, "aws4_request")
##     f) HMAC the string_to_sign with the key derived using steps a - e

vvar signature = ${string_to_sign}

iif (${DEBUG} == 1) {
    print ${banner}Deriving Signing Key using these parameters${banner}${NEWLINE}${secret_key} ${yyyyMMdd} ${AWS_Region} ${AWS_Service}${NEWLINE}${NEWLINE}
}}

# # The following statement takes care of all the details listed above
# # Notes: 
# #      - The word 'signature' in the statement below is the NAME of a variable and
# #        NOT a reference to its contents
# #      - The contents of this variable are the string to sign, and after the statement
# #        has completed these contents will have been modified to be the authorization
# #        signature for that string
#
AWS_sign_string signature using ${secret_key} ${yyyyMMdd} ${AWS_Region} ${AWS_Service}

#######################################################################################
## STEP 4                                                                             #
## Add the signing information to the request as documented at:                       #
## https://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html  #
##                                                                                    #
######################################################################################

vvar credential_scope = "${yyyyMMdd}/${AWS_Region}/${AWS_Service}/aws4_request"
iif (${DEBUG} == 1) {
    print ${banner} Credential Scope ${banner}${NEWLINE}${credential_scope}${NEWLINE}${NEWLINE}
}}

vvar auth_header = "Authorization: AWS4-HMAC-SHA256 Credential=${access_key}/${credential_scope}, SignedHeaders=${signed_headers}, Signature=${signature}"

iif (${DEBUG} == 1) {
    print ${banner} Authorization Header ${banner}${NEWLINE}${auth_header}${NEWLINE}
}}

sget http_header ${auth_header}

########################################################
## STEP 5                                              #
## Execute the query                                   #
##-----------------------------------------------------#
## Note that all the headers that were included in the #
## signed_headers created in STEP 1 must be set before #
## the request is executed                             #
########################################################

sget http_header "Date: ${date}"
sget http_header "x-amz-content-sha256: ${hashed_empty_string}"
sget http_savefile ${save_path}/${save_file}

sget http_progress yes
pprint "Downloading ${host}/${URI}:"
hhttp GET https://${host}/${URI}
pprint ${NEWLINE}Done